When a video camera is used for a measuring operation or a recognizing operation in a robot system, output images from an imaging device of the video camera are usually stored in a digital memory. An image processing apparatus receives an output from the digital memory and derives image coordinates which represent characteristic points, such as, for example, corner points of objects.
The image coordinates provided from the image process apparatus are two-dimensional coordinates associated with the digital memory and the imaging device. Since objects to measure or to recognize are in three-dimensional coordinates, to measure or recognize the objects requires conversion from the two-dimensional coordinates to the three dimensional coordinates. For the coordinate conversion, a mathematical camera model, which mathematically describes a production process of the image coordinates from the objects, is used. By using the mathematical camera model, coordinate values of the object in the three-dimensional coordinates can be obtained from the image coordinates in the two-dimensional coordinates.
Several mathematical camera models have been presented by researchers. Each mathematical camera model has common camera parameters, such as, for example, a position and an orientation of the camera (external parameters), a focal length and an image center (internal parameters).Generally, the camera parameters are determined by camera calibration. Since the derivation of coordinate values of the object is achieved by using the mathematical camera model, preciseness of measurement of the object is influenced by the camera parameters of the mathematical camera model. In other words, it is important for the camera calibration to determine these camera parameters precisely.
On the other hand, it is also important for the camera calibration to determine camera parameters instantly, especially in a real-time system. Generally, the camera parameters can be obtained quickly in a simple camera model. However preciseness of the camera parameters cannot be expected. On the other hand, in a complex camera model, the camera parameters can be obtained precisely but not quickly. Therefore, there is a contradiction in the mathematical model between precision and quickness.
To solve this contradiction, several camera models are proposed. For example, articles, such as, "A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-The-Shelf TV Cameras and Lenses" (R. Y. Tsai, IEEE Trans, Pattern Analysis and Machine Intelligence, Vol. 11, No. 5, pp451-496, 1987) and "Camera Calibration with Distortion Models and Accuracy Evaluation" (J. Weng et al, IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. 14, No. 10, pp965-980, 1992), propose non-linear camera models and "Two-Step Methods". In these articles, the non-linear camera models contain distortion models, which describe distortion provided by a camera system.
Generally, a non-linear system has difficulty obtaining an optimized solution; namely, to get precise camera parameters. This is because a non-linear system has a plurality of limit values in its evaluation function. The methods which are described in the two articles also have the same difficulties. Further, because the methods which are described in the two articles use simplified distortion models in their system, the solutions obtained thereby have limited preciseness. As described above, it is difficult to solve the contradiction present in the camera calibration by simply improving a camera model.